Devices with “always-on” or “always listening” voice interface capabilities, such as voice-enabled digital assistants, smart speakers, and hands-free interfaces traditionally require constant power, which either drains battery power or requires an electrical outlet. Portions of devices with speech recognition capabilities may remain in a low power consumption mode until speech like sounds are detected at which point phrase detection can determine whether a specific word or phrase has been spoken (i.e., a wake phrase). The implementation of wake phrase detection results in increased power consumption due to portions of the device remaining in a powered state over long periods of time (e.g., “always on”).